Singularly perturbed linear programs and Markov decision processes

Singularly Perturbed Markov Decision Processes: A Multiresolution Algorithm

Singular perturbation techniques allow the derivation of an aggregate model whose solution is asymptotically optimal for Markov decision processes with strong and weak interactions. We develop an algorithm that takes advantage of the asymptotic optimality of the aggregate model in order to compute the solution of the original model. We derive conditions for which the proposed algorithm has bett...

Asymptotic linear programming and policy improvement for singularly perturbed Markov decision processes

In this paper we consider a singularly perturbed Markov decision process with ®nitely many states and actions and the limiting expected average reward criterion. We make no assumptions about the underlying ergodic structure. We present algorithms for the computation of a uniformly optimal deterministic control, that is, a control which is optimal for all values of the perturbation parameter tha...

A State Aggregation Approach to Singularly Perturbed Markov Reward Processes

In this paper, we propose a single sample path based algorithm with state aggregation to optimize the average rewards of singularly perturbed Markov reward processes (SPMRPs) with a large scale state spaces. It is assumed that such a reward process depend on a set of parameters. Differing from the other kinds of Markov chain, SPMRPs have their own hierarchical structure. Based on this special s...

Geometric Interpretation of Hamiltonian Cycles Problem via Singularly Perturbed Markov Decision Processes

We consider the Hamiltonian cycle problem (HCP) embedded in a singularly perturbed Markov decision process (MDP). More specifically, we consider the HCP as an optimization problem over the space of long-run state-action frequencies induced by the MDP’s stationary policies. We also consider two quadratic functionals over the same space. We show that when the perturbation parameter, ", is suffici...

An asymptotic simplex method for singularly perturbed linear programs